The task of synthesizing a 3D scene from a 2D image is very difficult and complex. This is due, at least in part, to the fact that a significant amount of visual information is lost when a 3D environment is captured in a 2D image. One potential technique for addressing this loss of visual information can involve the use of depth images to synthesize a 3D scene. The depth images contain distance information relating to objects captured in the 2D image that can be used to synthesize the 3D scene. However, in many cases, depth images are not available and, therefore, cannot be used to synthesize the 3D scene.